Papers with ensemble techniques
Online Distilling from Checkpoints for Neural Machine Translation (N19-1)
Copied to clipboard
| Challenge: | Existing neural machine translation models have a deep structure with large amounts of parameters, making them hard to train. |
| Approach: | They propose an online method to generate a teacher model from checkpoints . they show steady improvement over a strong self-attention-based baseline system . |
| Outcome: | The proposed method improves on-the-fly on several datasets and language pairs. |